Part III: Deep Learning Research

• 14 Autoencoders
  • 14.1 Undercomplete Autoencoders
  • 14.3 Representational Power, Layer Size and Depth
  • 14.4 Stochastic Encoders and Decoders
  • 14.5 Denoising Autoencoders
  • 14.6 Learning Manifolds with Autoencoder
  • 14.7 Contractive Autoencoders
  • 14.8 Predictive Sparse Decomposition
  • 14.9 Applications of Autoencoders
• 15 Representation Learning
  • 15.1 Gready Layer-Wise Unsupervised Pretraining
  • 15.2 Transfer Learning and Domain Adaptation
  • 15.3 Semi-Supervised Disentangling of Casual Factors
  • 15.4 Distributed Representation
  • 15.5 Exponential Gains from Depth
  • 15.6 Providing Clues to Discover Underlying Causes
• 16 Structured Probablistic Models for Deep Learning
  • 16.1 The Challenge of Unstructured Modeling
  • 16.2 Using Graphs to Describe Model Structure
  • 16.3 Sampling from Graphical Models
  • 16.4 Advantages of Structured Modeling
  • 16.5 Learning about Dependencies
  • 16.6 Inference and Approximate Inference
  • 16.7 The Deep Learning Approach to Structured Probabilistic Models
• 17 Monte Carlo Methods
  • 17.1 Sampling and Monte Carlo Methods
  • 17.2 Importance Sampling
  • 17.3 Markov Chain Monte Carlo Methods
  • 17.4 Gibbs Sampling
  • 17.5 The challenge of Mixing between Seperated Modes
• 18 Confronting the Partition Function
  • 18.1 The log-likehood Gradient
  • 18.2 Stochastic Maximum Likehood and Contrastive Divergence
  • 18.3 Pseudolikehood
  • 18.4 Score Matching and Ratio Matching
  • 18.5 Denoising Score Matching
  • 18.6 Noise-Contrastive Estimation
  • 18.7 Estimating the Partition Function
• 19 Approximate Inference
  • 19.1 Inference as Optimization
  • 19.2 Expectation Maximization
  • 19.3 MAP Inference and Sparse Coding
  • 19.4 Variational Inference and Learning
  • 19.5 Learned Approximate Inference
  • Paper: Auto-Encoding Variational Bayes
• 20 Deep Generative Models
  • 20.1 Boltzmann Machines
  • 20.2 Restricted Boltzmann Machines
  • 20.3 Deep Believe Network
  • 20.4 Deep Boltzmann Machines
  • 20.5 Boltzmann Machines for Real-Valued Data
  • 20.6 Convolutional Boltzmann Machines
  • 20.7 Boltzmann Machines for Structured or Sequential Outputs
  • 20.8 Other Boltzmann Machines
  • 20.9 Back-Propagation through Random Operations
  • 20.10 Directed Generative Nets
  • 20.11 Drawing Samples from Autoencoders
  • 20.12 Generative Stochastic Networks
  • 20.13 Other Generation Schemes
  • 20.14 Evaluating Generative Models